On Soft-Biometric Information Stored in Biometric Face Embeddings

The success of modern face recognition systems is based on the advances deeply-learned features. These embeddings aim to encode identity an individual such that these can be used for recognition. However, recent works have shown more information beyond user’s stored in embeddings, as demographics, image characteristics, and social traits. This raises privacy bias concerns We investigate predictability 73 different soft-biometric attributes three popular with learning principles. experiments were conducted two publicly available databases. For evaluation, we trained a massive attribute classifier accurately state confidence its predictions. enables us derive sophisticated statements about predictability. results demonstrate majority investigated are encoded embeddings. instance, strong encoding was found haircolors, hairstyles, beards, accessories. Although robust against non-permanent factors, specifically easily-predictable from hope our findings will guide future develop privacy-preserving bias-mitigating technologies.